Papers by Hai An Vu
EMO: Embedding Model Distillation via Intra-Model Relation and Optimal Transport Alignments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation focus on direct output alignment, neglecting this crucial structural information. |
| Approach: | They propose a framework for knowledge distillation that maps tokens one-to-one and aligns attention matrix patterns using Centered Kernel Alignment. |
| Outcome: | The proposed framework significantly outperforms existing CTKD baselines. |
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining (2026.findings-acl)
Copied to clipboard
Phung Gia Huy, Hai An Vu, Minh-Phuc Truong, Thang Duc Tran, Linh Ngo Van, Thanh Hong Nguyen, Trung Le
| Challenge: | Existing approaches to train dense representations require explicit coordination of how information is arranged across embedding dimensionality and model depth. |
| Approach: | They propose a framework that trains Matryoshka representations using self-distilled intra-relational alignment and Progressive information chaining. |
| Outcome: | The proposed framework produces coherent and compact Matryoshka representations with significant performance advantages under low-dimensional models. |